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ABSTRACT 

The Pilot's ability to derive Control-Oriented 
Visual Field Information from teleoperated Helmet- 
Mounted displays in Nap-of-the-Earth flight, is 
investigated. The visual field with these types of dis- 
plays, commonly used in Apache and Cobra helicopter 
night operations, originates from a relatively narrow 
field-of-view Forward Looking Infrared Radiation 
Camera, gimbal-mounted at the nose of the aircraft and 
slaved to the pilot's line-of-sight, in order to obtain a 
wide-angle Field-of-regard. Pilots have encountered 
considerable difficulties in controlling the aircraft by 
these devices. Experimental simulator results presented 
here, indicate that part of these difficulties can be 
attributed to head/camera slaving system phase lags 
and errors. In the presence of voluntary head rotation, 
these slaving system imperfections are shown to 
impair the Control-Oriented Visual Field Information 
vital in vehicular control, such as the perception of the 
anticipated flight path or the vehicle yaw rate. Since, 
in the presence of slaving system imperfections, the 
pilot will tend to minimize head rotation, the full 
wide-angle field-of-regard of the line-of-sight slaved 
Helmet-Mounted Display, is not always fully utilized. 


INTRODUCTION 

With head-slaved Helmet-Mounted Displays 
(HMD's), the image of a forward looking camera, such 
as an Infrared Radiation or a low-light level camera, 
mounted on a servo-driven gimbals system at the front 
of the helicopter, is transferred to a miniature helmet 
mounted Cathode Ray Tube (CRT). By means of 
collimating optics and a beam splitter, the image is 
presented to a single eye so that it appears to be 
superimposed on the visual field at infinity. The 
camera motions are slaved to the pilot's Line-of-Sight 
(LOS), by measuring the pilot’s head angles in pitch 
and yaw, and by imparting this information to the 
camera servo drives. The LOS slaving system of 
HMD's allows the field-of-regard of the pilot to be 
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extended well beyond the limits of the narrow field-of- 
view of the HMD's viewing optics. Thus, just by 
rotation of the head, the pilot is able to cover a field- 
of-regard of nearly up to 180 deg horizontally, and 90 
deg vertically. In addition, in most cases, the camera is 
positioned such that its view is not obstructed by the 
aircraft body. This allows the pilot to view areas 
which are usually blocked out by the cockpit. This 
wide-angle coverage is very essential in Nap-of-the- 
Earth (NOE) flight, both for vehicular control by 
allowing the pilot the necessary spatial orientation 
with respect to terrain and obstacles, and for the 
detection and location of targets or mission threats in 
military missions, or survivors in rescue missions. 

Although the LOS slaved HMD apparently solves 
the problem of providing a wide-angle coverage for the 
given narrow field-of-view of the HMD optics, vehic- 
ular control with such systems is still very difficult, 
and demands high pilot proficiency and work load. Part 
of these difficulties can be attributed the fact that the 
viewpoint of the camera is displaced with respect to 
the actual eye position. In the presence of fast vehicle 
pitch or yaw rotations this might result in misjudged 
vehicle motions. Furthermore, for a camera mounted 
in front of the pilot, near objects will appear larger 
than they actually are. 

Additional difficulties arise from the relatively 
narrow field-of-view of the HMD's viewing optics, re- 
sulting from practical limitations on the miniature 
CRT face-plate dimensions, the dimensions and shape 
of the beam splitter and its minimal safe distance to 
the pilot's eye, and the collimating system design. 
Thus, essential parts of the pilot’s peripheral vision 
are missing, which may result in impaired motion 
perception. 

Considerable difficulties are also encountered in 
the interpretation of FLIR images, which are basically 
different from visible light images, usually resulting 
in misjudged object size and impaired depth 
perception. 

The head-slaved HMD resembles a viewing 
aperture without optics, attached to the pilot's head, 
which allows him to frame-in different areas of the 
outside world by rotation of the head. The HMD one- 
to-one slaving system and the deliberate choice of a 
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unity image magnification, attempt to give the pilot 
the illusion of viewing a natural visual scene through 
such an aperture. For an ideal slaving system, the 
viewed image would appear to be part of an inertially 
stable background. However, slaving system errors 
will be experienced by the pilot as undesired shifts of 
the displayed visual field with respect to the true 
"natural" visual field. The effect of these shifts is two- 
fold: (1) they will alter the optical flow-field pattern 
and result in incorrect estimation of the self-motion; 
and (2) the visually estimated self-motion will be 
different from the motion estimated by vestibular cues. 
This might lead to visuo-vestibular conflicts or 
motion sickness, Oman [1], In coordinated fixed-wing 
aircraft flight, the velocity vector will coincide with 
the vehicle longitudinal axis and the pilot can infer the 
direction of motion from vehicle-based references, by 
means of his kinesthetic sense of straight ahead. 
However, in helicopter flight, the direction of motion 
can deviate substantially from the vehicle axis. Thus 
helicopter control in NOE flight, is susceptible in 
particular to these slaving system imperfections, since 
the Control-Oriented Information has to be derived 
entirely from the visual field, and can not rely on 
vehicle based references. 

This paper deals with the basic experiments for 
understanding the Visual Field Information in HMD's 
Displays, and investigates how this information is 
affected by slaving system imperfections. Two types 
of experiments were carried out: (1) a flight path 
estimation experiment, in which the pilot had to judge 
the anticipated vehicle path, while being flown 
passively in a straight or curved horizontal path over 
flat textured terrain, and (2) a simulated Nap-of-the- 
Earth flight experiment, in which the pilot subject had 
to fly actively through a winding canyon, in the 
presence of purposefully induced head motions. 


VISUAL FLOW FIELD CUES 
A detailed geometrical analysis of visual flow 
field cues in horizontal flight over textured flat terrain, 
is given in Ref. [2], In this paper we shall suffice with 
a brief qualitative description. 

The visual flow field resulting from an observer’s 
self-motion, is given by the time derivative of a set of 
line-of-sight (LOS) vectors extending from the pilot's 
eye to conspicuous points in the visual field (texture 
points). The flow field is the pattern traced by the 
intersection of these LOS vectors with a unity sphere 
about the observer's head. These traces are commonly 
referred to as the "streamer" pattern. For straight or 
constantly curved motion at fixed velocity and altitude 
above a flat surface, the flow field is constant. 


Flow field cues in straight flight 

The horizontal situation for straight and level 
flight is shown in Fig. la. The center of gravity of the 
vehicle moves along a straight path in the direction of 
the velocity vector V, while the longitudinal vehicle 
axis is xb is rotated with respect to V by the crabbing 
angle (J. The camera axis xh is rotated with respect to 
the vehicle axis by the angle \j/h- The corresponding 
streamer pattern is shown in Fig. lb. The horizontal 
and vertical axes in Fig. lb are the viewing azimuth 
angle and elevation angle, (the latter is measured 
positive in upwards direction). 

For straight flight the streamer pattern appears to 
expand from a common focal point on the horizon, 
point F, see Fig. lb. This point has often been called 
the "focus of expansion", Gibson [3-5]. The straight 
vehicle path is defined by the set of points which do 
not have an azimuth LOS rate component, see solid 
line. This is also the streamer that is apparently 
vertical, i.e. perpendicular to the horizon or to the base 
of the HMD image frame, for zero vehicle roll angle. 
The dotted box in Fig. la indicates the area of the 
visual field, viewed by the HMD. The center of this 
box, H, indicates the camera axis x^ and coincides 
with the pilot's direction of gaze. The vehicle 
longitudinal axis Xb is indicated by point C. The head 
angle yh is the angle between H and C, and the 
vehicle crabbing angle (3 is the angle between F and C. 
In case the crabbing angle p is zero, F and C coincide 
and the direction of motion is presented to the pilot 
implicitly by kinesthetic head position cues. However, 
for arbitrary large angles of p this is not the case, and 
the direction of motion has to be derived solely from 
the streamer pattern. 

Fig. lb. indicates that the focal point F is not 
necessarily located within the HMD field-of-view. In 
this case, the direction of motion is derived by 
estimating the point where the streamers line 
segments, visible within the HMD viewing area, 
would intersect. It has been shown in Ref. [2] that the 
detectability of the direction of motion depends on the 
local expansion, which is defined as the derivative of 
the streamer direction with respect to the azimuth 
angle. This local expansion is shown to be 
proportional to the viewing distance to the texture 
point, measured along the LOS. It therefore appears, 
that the direction of motion is most easily perceived in 
the far visual field, where the local expansion is the 
largest. However, the streamer pattern can only be 
perceived when the magnitude of the LOS rates are 
above a certain threshold. It is shown in Ref. [2] that 
these LOS rates are inverse proportional to the squared 
viewing distance. A possible mechanism for 
estimating the straight vehicle path is to extrapolate 
the focal point from converging streamer segments, 
located within an area of the visual field, at the farthest 
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viewing distance at which the streamer direction can 
still be detected, and at which the local expansion is 
the largest. It is clear that when this area is not within 
the HMD field-of-view, the pilot will have to shift his 
gaze to a different area of the visual field. 

Flow field cues in curved flight 

The horizontal situation for steady curved flight 
over flat terrain is shown in Fig. 2a. The 
instantaneous velocity vector is X> and P is again the 
crabbing angle. However, the actual vehicle path is a 
circle with radius R, tangential to X and with its 
center at point M. The corresponding streamer pattern 



STRAIGHT MOTION VELOCITY VECTOR 



Figure L (a) Horizontal Situation for Straight and 
Level Flight; (b) Streamer Pattern for Straight 
and Level Flight over Flat Textured Terrain 


is shown in Fig. 2b. This pattern shows a converging, 
curved set of lines, and a common focal point on the 
horizon no longer exists. The curved vehicle path is 
the dashed, central line in the bundle. The vehicle path 
is defined by the streamer which, for very close 
viewing ranges, will have a zero azimuth LOS rate 
component and which tend to be tangential to the 
velocity vector (solid vertical line). This tangent is 
again apparently vertical, i.e. perpendicular to the 
horizon or to the base of the HMD image frame, for 
zero vehicle roll angle. 



CURVED MOTION velocity vector 

vehicle path 
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Figure 2. (a) Horizontal Situation for Curved Level 
Flight; (b) Streamer Pattern for Curved Level 
Right over Flat Textured Terrain 
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It is of interest to consider points in the visual 
field, which do not have an azimuth LOS rate 
component. It is shown in Ref. [2] that the locus of 
these points is formed by the circle, tangential to Y. 
and with radius 0.5R , hereafter referred to as the "half- 
radius circle". This locus is shown in Fig. 2b as the 
dotted line. For viewing distances D«R, the azimuth 
angle of a point on the vehicle path is about half way 
in between the azimuth angle of the velocity vector 
and of a point on the half-radius circle. 

A possible mechanism for estimating a point on 
the vehicle path, would look for the azimuth angle of 
the area at viewing distance D, with a zero azimuth 
LOS rate component, (on the half-radius circle). In 
addition, the mechanism would estimate the azimuth 
of the velocity vector by looking, at very close 
distances, for points with a zero azimuth LOS rate 
component. It would then estimate the azimuth of a 
point on the vehicle path at distance D to be half way 
in between the two angles. A shortcoming of this 


mechanism is that it will break down when the point 
on the half-radius circle is outside the field-of-view. 

Another possible mechanism would be to look for 
continuity of motion between pom is in the visual 
field. It would select a set of points which belong to a 
certain section of the streamer, by following the 
motion of a texture point over a given interval of 
time. It would then find the correspondence between 
streamer sections which would add up to the central 
streamer, i.e. the one of which the azimuth LOS rate 
component for close viewing distances, is zero, or, 
alternatively, the streamer which tends to be tangential 
to the apparent vertical. This would involve viewing 
near as well as far areas of the visual field. 

The dotted box in Fig. 2b. again shows the area 
of the visual field, viewed by the HMD. Regardless of 
the mechanism used, active head motions of the pilot 
will be required, since the estimation of the curved 
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Figure 3: Streamer Pattern "Bending" Effect Resulting From Line-of-Sight Slaving System Lags 
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vehicle path is based on different areas of the visual 
Field, which involve either points on the half-radius 
circle or points nearby. 

Flow field cues in helmet mounted displays 

As mentioned before, for an observer in straight 
or steady curved level flight, a stationary streamer 
pattern is obtained. Under natural, unconstrained 
viewing conditions, the streamer pattern and 
consequently the ability to estimate the vehicle path, 
will not be affected by voluntary head rotations. This 
can be attributed to the visuo-ocular reflex, which will 
inertially stabilize the eye line-of-sight with respect to 
the viewed background. In this situation, changes in 
the pattern only result from changes in self-motion 
parameters. However, the viewing conditions for LOS 
slaved HMD's differs from natural unconstrained 
conditions in two ways: (1) the narrow viewing 
aperture and (2) slaving system imperfections. Due to 
the narrow viewing aperture, areas essential for 
estimating the vehicle path might not be in view. 
Attempts of the pilot to acquire the information might 
require quick scans of different areas of the visual Field, 
resulting in rapid head motions. However, due to the 
frame-of-reference effect caused by the narrow viewing 
aperture, the observer might experience an apparent 
yaw motion in opposite direction of the voluntary 
head rotation, Boff [6], This illusion is caused by 
strong edge rate effects of image elements passing the 
edge of the HMD image frame during rotation. The 
smaller the reference frame, the stronger the effect. 

Slaving system imperfections are detrimental in 
particular in perceiving self-motion information from 
the visual field. Random tracking errors, scaling 
errors, or phase lags will result in undesired shifts of 
the displayed visual Field with respect to the true 
"natural" visual field. These undesired image shifts 
will make the viewed visual scene appear to move 
with respect to an inertially stable background. The 
negative effects of this apparent motion are twofold: 
(1) Since, during voluntary head rotation, the eye LOS 
is stabilized with respect to inertial space, the parasitic 
image shifts will alter the visual Field information. (2) 
The self-motion estimated from the shifted visual Field 
is in conflict with the vestibular signals. This visuo- 
vestibular conflict might cause motion sickness or 
disorientation. The following example demonstrates 
the effect of parasitic image shifts due to LOS slaving 
system servo lags, on the visual field information 
contents. 

Consider the LOS slaving system to be a second- 
order system with an natural frequency of co n =94.3 
rad/s (15 Hz) and with a damping factor of £=0.707. 
Consider the head yaw rotation to be sinusoidal with 
amplitude A deg and frequency co rad/s. It is easily 
shown that for oK<o>n the image shift rate amplitude 


s is given by: s==2£Aco 2 /co n deg/s. 

For example, for A=10 deg and o>=1.0 rad/s the 
image shift rate amplitude is s=0.15 deg/s. This 
parasitic yaw rate will add a constant azimuth 
component to all LOS vector rates. For an observer in 
straight motion the parasitic yaw rate will make the 
expanding pattern appear to "bend" momentarily, just 
as if the observer were in curved motion. It is clear 
that the larger the ratio between parasitic yaw rate and 
LOS rate, the larger the "bending" effect. Therefore the 
negative effects of servo lags are noticed in particular 
for low self-motion velocities. Fig. 3 shows examples 
of this "bending" effect, for A=10 deg and co=1.0 rad/s, 
for various velocities. Both the velocity V and the 
viewing distance D are expressed in units of the height 
h above the terrain. For V/h ratios of 0.25 s and 4.0 s, 
the angular errors introduced by the bending effect at 
viewing distance D=7.5h are 2.26 deg and 0.14 deg, 
respectively. 


EXPERIMENTAL PROGRAM 
Experimental setup 

The visual scene was generated at a Silicon 
Graphics IRIS 4D 50/GT work station. The pilot 
subject was seated in a general aviation simulator 
cabin, wearing an operational flight helmet, on which 
a Hughes Aircraft miniature CRT with beam splitter 
and collimating optics was mounted. The 
monochromatic image, of aperture 22.8 deg 
horizontally and 18.4 deg vertically, was presented to 
the subject's left eye only. The right eye was 
uncovered, viewing the low-light level cockpit 
background, normally present in night helicopter 
missions. No outside view or panel mounted display 
images were presented. A Polhemus head tracking 
system was used to measure the angular orientation of 
the head. The measured yaw, pitch and roll head angles 
were send to the graphics work station, and used for 
generating the image corresponding with the subject’s 
line-of-sight. Although the Polhemus was sampling at 
30 Hz, the image was updated at about 15 Hz. Thus, 
the system roughly simulated a line-of-sight slaving 
system with a bandwidth of 15 Hz. Pilot controls 
included a two-axis high-precision strain gauge 
operated side-arm controller with response buttons. 

Flight path estimation experiment 

Each trial represented the situation of passively 
being flown over nominally flat terrain at height h, 
either in a straight or constantly curved, level motion 
pattern. The terrain consisted of a field of randomly 
placed poles with constant density and with no visible 
alignment. The average distance between the poles was 
1.17 units of h, and their average height was 0.25h. 
Both the vehicle velocity vector V, and the vehicle 
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longitudinal axis xb were parallel to the ground plane. 
The vehicle axis was deviating from the velocity 
vector V by the crabbing angle P, and the curved path 
was tangential to V_. In order to conserve 
computational resources and realize an update rate of 
15 Hz, the field was not drawn beyond a viewing 
distance of D=I5h. 

The subjects initiated an experimental trial by 
pressing a response button, after which the visual field 
became visible from an initially blank screen. For 
each trial the side slip (crabbing) angle P and/or the 
path curvature radius R were uncorrelated and chosen 
randomly. A marker was visible in the visual field at 
viewing distance D, in the direction of gaze, 
consisting of a circular base of diameter 0.625 h placed 
in the ground plane with a vertical pole at its center of 
height 0.125 h. The marker remained at the center of 
the HMD image, and the subjects could change the 
marker azimuth just by turning their head. It should be 
noted that through appropriate geometrical 
transformations, the marker was kept at all times 
perpendicular to the ground plane and at a fixed 
viewing distance D, regardless of head pitch and roll. 
The subjects were asked to place the marker on the 
estimated flight path. They were instructed to do this 


intuitively, as quickly as possible and to acknowledge 
their choice by pressing a response button. During the 
training runs, after each trial, a dotted line was 
displayed for two additional seconds, indicating the 
true flight path. 

Three types of experiments were conducted: (1) 
Straight and level flight in the presence of a constant 
side slip angle p, chosen from a uniformly distributed 
random set, ranging from -45 to +45 deg. (2) Steady 
curved and level flight with zero side slip, where the 
path curvature radius was chosen from a uniformly 
distributed random set ranging from 15h to 40h and 
where the curvature could be to the left or to the right 
with equal probability, and (3) Steady curved and level 
flight in the presence of side slip, with the curvature 
chosen as in (2) and with the side slip angle P chosen 
from an uncorrelated uniformly distributed random set, 
ranging from -14 to +14 deg. 

The relevant parameters investigated were: the 
velocity-to-height ratio and the viewing distance. Five 
velocity-to-height ratios were chosen, ranging from 
0.25s to 4s. Two viewing distances were chosen, 
D=7.5h for the far field, and D=3.0h for the near field. 



Figure 4: Image of the Randomly Curved Canyon used in the Simulated NOE Flight Experiments 
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Simulated NOE flight experiment 

This experiment simulated the task of actively 
flying a control-augmented H-I9 helicopter through a 
V-shaped randomly curved canyon, shown in Fig. 4. 
The horizontal canyon path, and the vertical path 
profile were generated by passing band-limited white 
noise processes through a series of shaping filters. The 
horizontal and vertical path correlation length was 
about 1500 ft Two trajectory shapes were considered: 
a "moderately" curved trajectory with maximum 
horizontal curvature radii of 500 ft and a strongly 
curved trajectory with maximum radii of 250 ft. For 
each run, lasting 180 s, a different random path was 
generated. The canyon was formed by randomly shaped 
cross sections spaced 30 ft apart and interconnected by 
monochrome, solidly drawn polygons of randomly 
different brightness, simulating a FLIR image. The 
subjects were instructed to follow the canyon while 
staying as close as possible to its base without hitting 
the sides. 

Deliberate head rotation was introduced by a 
secondary task. At random intervals, a diamond-shaped 
target appeared at a location, fixed with respect to the 
canyon, see Fig. 4. The target became first visible at a 
viewing range of 550 ft and disappeared at 300 ft The 
subject had to lock his line-of-sight on the target, by 
bringing it within a 5.7 by 5.7 deg tick mark area in 
the center of the image. After a successful lock-on, the 
target and tick marks disappeared. During each run a 
total of 15 targets were presented. The target locations 
were chosen such that they involved considerable head 
rotation, in addition to the rotation needed for 
following the canyon. 

Subject training and experimental procedure 

Eight male and one female subject, all of them 
Technion Aerospace undergraduate students, 
participated in the experiment. Subject age was 
between 19 and 24. Subject training for the vehicle 
path estimation experiments included several one-hour 
training sessions. After that each subject carried out a 
series of runs for each one of the three experiments 
(straight, curved, curved with side slip, in this order). 
Each series included a number of configurations, each 
of which was repeated four times and addressed in a 
random fashion. Each configuration consisted of a set 
of 20 consecutive trials, each of which was initiated 
by the subject by pressing a response button. Each 
trial lasted for about 2-8 seconds, depending on the 
time needed by the subject to estimate the direction of 
motion. About 8 one-hour sessions were needed for 
each subject to finish the experimental program. 

Training for the simulated NOE flight experiment 
required several one hour sessions. Production included 
simulation runs of 180 s duration, repeated 5 times for 
each subject and for each configuration. Subject 


motivation was enhanced by a reward system based on 
competition. 

Experimental measurements 

In the flight-path estimation task, for each trial in 
a set, the error in azimuth angle between the true and 
estimated location of a point on the flight path at 
viewing distance D, were recorded, together with the 
time needed to make the estimate. The upper limit on 
the estimation time was 8 seconds, after which the run 
was terminated and marked as a failed run. In addition, 
the head activity was recorded in terms of the standard 
deviation of the head yaw angle and yaw angle rate. 

For each set of 20 trials, the average and the 
standard deviation of the estimation error and 
estimation time, were computed. Since the average of 
the estimation error was found to be almost zero, i.e. 
no preference for an error in left or in right direction 
existed, the standard deviation of the estimation error 
was adopted as the representative estimation error score 
of each set. For the estimation time, the average of the 
set was taken as the representative score. 

Performance scores for the NOE flight 
experiments included the power of the deviation from 
the bottom of the canyon, standard deviations of head 
activity, stick activity, vehicle roil and roll rate and 
the average time needed to lock the line-of-sight on the 
target. 

EXPERIMENTAL RESULTS 
Flight path estimation task 

Effect of the velocity-to-height ratio: 

Figs. 5-8 show the various performance scores as 
a function of the V/h ratio, for the three motion 
patterns. For straight motion, (dotted line) the 
estimation error score strongly decreases with V/h, 
both for the "far" viewing distance of D/h=7.5h (Fig. 
5a) and for the "near" viewing distance of D/h=3.0h 
(Fig. 5b). In contrast, the downslope of the curves for 
curved motion and curved motion with side slip, is 
considerably less. For the near viewing distance the 
curves are even sloping upwards (Fig 5b). 
Furthermore, the curves for straight motion for the 
estimation time and the head yaw rate activity are 
markedly above the ones for the curved motion 
patterns, see Figs. 6a, b. This indicates that the 
subjects probably used a different strategy in the 
straight motion task. The curves for the head yaw 
angle and yaw angle rate activity show pronounced and 
consistent upward slopes, Figs. 7 and 8. This effect 
can be attributed to the LOS slaving system 
imperfections, discussed in Section 2.4. The smaller 
V/h, the stronger the "bending" of the streamer pattern 
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ESTIMATION TIME [SEC] ESTIMATION ERROR SCORE [DEG] 



0.25 0.50 1.00 2.00 4.00 

VELOCITY TO HEIGHT RATIO [SEC - '] 


Figure 5a. Estimation Error Score for D=7.5h 



0.25 0.50 1.00 2.00 4.00 


VELOCITY TO HEIGHT RATIO [SEC - '] 
Figure 6a. Estimation Time for D=7.5h 



0.25 0.50 1.00 2.00 4.00 


VELOCITY TO HEIGHT RATIO [SEC -1 ] 
Figure 5b. Estimation Error Score for D=3.0h 



0.25 0.50 1.00 2.00 4.00 


VELOCITY TO HEIGHT RATIO [SEC -1 ] 
Figure 6b. Estimation Time for D=3.0h 
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HEAD YAW RATE ACTIVITY [DEG/SEC] HEAD YAW ANGLE ACTIVITY [DEG] 



Figure 7a. Head Yaw Angle Activity for D=7.5h 



Figure 7b. Head Yaw Angle Activity for D=3.0h 



Figure 8a. Head Yaw Rate Activity for D=7.5h 



Figure 8b. Head Yaw Rate Activity for D=3.0h 


323 





ESTIMATION TIME [SEC] _ ESTIMATION ERROR SCORE [DEG] 


8.0 -i 


8.0 A 


4.0 A 


2.0 


0.0 


-1 


V/H 1.0 SEC D/H 7.S] 
ES NOMINAL 
PHASE LAG 
REDUCED FOV 




si 


Si 




STRAIGHT CURVED CURVED + 
SIDESLIP 


r-— i 3.0 

o 

LU 

Q 

| 6.0 

h- 

CJ 

< 

LU 


o 


£ 

a 

< 

LU 


4.0 A 


2.0 A 


0.0 


V/H 


1.0 SEC D/H 7.5 
53 NOMINAL 
PHASE LAG 
REDUCED FOV 






i 






i 



STRAIGHT CURVED 


CURVED + 
SIDESLIP 


Figure 9a. Effect of Reduced Field-of-View and 
LOS Slaving Lags on Estimation Error Score 


Figure 9c. Effect of Reduced Field-of- View and 
LOS Slaving Lags on Head Yaw Angle Activity 
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Figure 9b. Effect of Reduced Field-of-View and 
LOS Slaving Lags on Estimation Time 


Figure 9d. Effect of Reduced Field-of-View and 
LOS Slaving Lags on Head Yaw Rate Activity 
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during head rotation resulting from slaving system 
lags, and the less accurate the perception of the 
streamer direction. This is detrimental in particular in 
the straight motion task, in which the streamer 
"bending" makes it almost impossible to find the 
apparent vertical streamer. As a result, for small 
velocities subjects will minimize their head rotation, 
estimates will take longer and estimation errors will 
be larger. This explains the strong downward slope of 
the estimation error curve for straight motion, as 
compared to the more flat curves for curved motion, 
see Fig. 5a. 

Strategy differences between straight and curved 
tasks are apparent when considering that the subjects 
are expecting a straight expanding motion pattern in 
the straight task. Thus, bending effects due to slaving 
system lags will be identified immediately, and 
estimates are made only at moments at which the head 
is stationary. The subjects might have employed a 
"null measurement" method, in which a sequence of 
correcting steps is made aimed at placing the apparent 
vertical streamer at the center. In contrast, in the 
curved task the subjects might have employed a 
"deflection measurement" method in which the vehicle 
path is found intuitively, more or less in an open 
manner, triggered by the amount of streamer "bending" 
in the field. Consequently, estimation times and head 
yaw activity are much larger for the straight task. 

Effect of side slip on curved motion: 

The estimation error score curves for motion with 
and without side slip have similar characteristics, i.e. 
for the far viewing distance a minimum at V/h=1.0 s 
(Fig. 5a), and for the near viewing distance similar up 
slopes, (Fig. 5b). This up slope might be due to 
motion "blurring" effects, which prevent the subject 
from making accurate estimates on the streamer 
pattern direction. However, the error score curve for 
motion with side slip is on the average about 3 deg 
above the one for motion without side slip. This was 
expected, since in the first case, in the process of 
estimating the vehicle path, the observer has to derive 
the direction of motion from the near visual field, 
whereas in the latter case the direction of motion is at 
zero azimuth. This direction is presented to him 
implicitly by kinesthetic head position cues. The 
increased difficulty to estimate the vehicle path in the 
presence of side slip is also noticed in the higher head 
yaw angle and yaw angle rate activity, see Figs. 7 and 
8 . 

Effect of the viewing distance: 

A comparison of the curves of Figs. 5a and 5b 
shows that the near viewing distance yields generally 
larger estimation errors than the far distance. This 
might result from the smaller local expansion in the 
near field. The difference is large in particular for high 


V/h ratios, probably as a result of image blurring. In 
contrast, the head yaw rate activity shown in Fig. 8a,b 
for the near and far viewing distance were found to be 
very similar. It would be expected that the near field, 
with its higher LOS rates, would allow larger head 
rotation, since less streamer "bending" will occur. 
However, the negative effect of the "bending" will be 
stronger, due to the smaller local expansion. 
Therefore, the subjects will still minimize their head 
rotation for low V/h and for the near viewing distance. 

Effect of reduced field-of-view: 

The results for an HMD field-of-view reduced to 
13.7 deg horizontally and 11.0 deg vertically (40% 
reduction), are shown in Fig. 9. Contrary to what was 
expected, estimation errors for all three motion 
patterns were about the same as for the nominal 
viewing situation. Fig 9a. However, estimation times 
were slightly higher (by 9%), and head yaw rate 
activity lower (by 15%), in particular for straight 
motion, Fig 9b, c. This indicates that although the 
reduced field-of-view did not affect estimation accuracy, 
the subjects might have reduced their head yaw rates 
due to increased edge rate effects. On the other hand, as 
expected, the reduced field-of-view demanded slighdy 
more headmotions, as seen in the 6.5% increase in 
head yaw angle activity 

Effect of LOS slaving system lags: 

A first-order slaving system lag with a time 
constant of 0.5 s was introduced. Although the phase 
lag yielded an only 4% higher error score as compared 
to the nominal viewing situation, the head yaw rate 
activity was markedly smaller (by 53%) and, 
consequently, the estimation time higher (by 30%), 
see Fig. 9. This clearly demonstrates that slaving 
system phase lags primarily constrain the subject from 
making fast head motions, they require from him to 
make more corrections (14% larger head yaw angle 
activity), and they result in longer estimation times. 

Simulated NOE flight task 

Results for the NOE flight task are summarized in 
Table I. For flight without target capture secondary 
task, the increase in path curvature has its primary 
effect on the tracking performance (a 65% increase in 
tracking error score). The increased path curvature is 
also strongly noticed in the 31% larger head yaw angle 
rate. Thus, the high-curvature canyon demands more 
head activity, which, in the presence of inherent LOS 
slaving system lags, adversely affects tracking 
performance. As expected, the increased curvature also 
yields larger control activity and vehicle roll motions. 

The effect of adding the target capture secondary 
task to the vehicular control task is strongly noticed 
both in the markedly higher tracking error and in the 
larger head yaw rates (tracking error scores increase by 
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Table 1. Simulated NOE Flight Experiment 



Without Target 
Task 

With Target 
Task 


Low 

Curva- 

ture 

High 

Curva- 

ture 

Low 

Curva- 

ture 

High 

Curva- 

ture 

Tracking 
Error [ft 2 ] 

390.1 

643.9 

1168.6 

1072.5 

Head Yaw rate 
[deg/s] 

3.2 

4.2 

7.4 

8.4 

Comm. Roll 
rate [deg/s] 

25.7 

27.7 

27.2 

29.1 

Comm. Pitch 
rate [deg/s] 

2.6 

3.1 

3.2 

3.7 

Vehicle Roll 
Angle [deg] 

7.5 

9.4 

8.8 

10.7 

Vehicle Roll 
Rate [deg/s] 

15.6 

16.7 

16.3 

17.5 

Target Cap- 
ture Time [s] 

- 

- 

1.48 

1.94 

% Missed 
Targets/Run 



6.6 

13.0 


116% and head yaw rates by 113%). In contrast, 
control activity and vehicle roll activity increase only 
slightly. The high correlation between head yaw 
activity and tracking errors, again demonstrate that 
head rotation, in excess to the amount needed for the 
vehicular control task, negatively affects performance. 
As expected, due to increased main task difficulty, the 
high-curvature canyon resulted in a 31% higher 
average target capture time and yielded twice as much 
missed targets. 

Although in this experiment the head rotation is 
artificially and purposefully induced, similar 
performance degradation is expected to occur, when the 
pilot voluntarily moves his head in search of targets or 
mission threats. Consequently, for HMD’s subjected 
to line-of-sight slaving system lags, the pilot will 
tend to reduce his head rotation to the minimum 
required for carrying out the vehicular control task. 
However, under these conditions, the wide-angle 
coverage of the line-of-sight slaving system will not 
be fully utilized and target search performance and 
spatial orientation will be seriously impaired. 


Summary of results 

1. The experimental results have clearly shown 
that line-of-sight slaving system imperfections in 
HMD's seriously impair the pilot’s ability to derive 
Control-Oriented Information from the visual field. 
Since, under these conditions, the pilot will tend to 
minimize head rotations, the wide-angle coverage 
provided by the slaving system, will not be utilized 
and search performance and spatial orientation will be 
impaired. 

2. Canyon following performance was found to 
deteriorate with increased head rotation, either when 
introduced in a "natural manner" through a higher path 
curvature, or when induced purposefully by using the 
target capture secondary task. 

3. The vehicle path estimation accuracy and head 
yaw rate activity generally increase with the V/h ratio. 
Due to the larger "local expansion" the far viewing 
distances yield more accurate estimates than close 
distances. However, due to blurring effects, close 
distance estimates no longer improve with V/h. 

4. The flight path for curved motion is 
considerably more difficult to estimate than for straight 
motion, since it relies on the entire streamer pattern 
rather than on local field estimates. Since in curved 
flight the near as well as the far field is used, the 
estimates are less accurate and improve less with 
increasing V/h ratio. 


DISCUSSION 

The display system discussed in this paper can be 
classified as a virtual environment display. Head- 
mounted displays have become a vital component of 
virtual environments, which attempt to give the 
operator the illusion of being physically present in a 
remotely existing or synthetically generated world. 
Frequently this objective is achieved by fully 
immerging the operator in the visual scene by 
completely blocking out the direct view of the outside 
world and by presenting the operator with a stereo 
image of the environment, which is derived either 
from a remotely located stereo camera pair or computer 
generated. Although state-of-the-art miniature display 
technology and computer generated image techniques 
enable to display images of high quality, detail and 
authenticity, most system fail to provide the operator 
with the confidence to move around freely without the 
fear of stumbling or falling. 

The main findings in this paper are valid for this 
general class of displays as well. While designers of 
virtual environments are devoting considerable 
attention to picture contents, quality and detail, the 
dynamic aspects are often neglected. Detrimental 
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factors are insufficient update rates, too large time 
delays due to time-consuming signal communication 
or highly band limited camera slaving systems. Other 
factors are inaccurate head position measurements and 
a lack of rigidity between the display and the head. 
While these displays may be adequate for a seated 
person in a near-static environment, in the presence of 
slow head motions, they often fall short in situations 
in which self-motion estimation is essential, such as 
walking, running or controlling a vehicle. Since 
correct motion estimation from visual cues is only 
possible when the illusion of an inertially stable 
background is preserved, deviations induced by system 
lags or slaving system errors, will result in estimation 
errors in the self-motion variables. Furthermore, for 
the person immerged in the environment, the visual 
cues will be in conflict with the vestibular ones, 
resulting in disorientation, loss of balance or even 
motion sickness. 

The display, discussed in this paper provides only 
"partial immersion" since the outside world remains 
directly visible both to the uncovered eye and to the 
covered one through the beam-splitter. This 
arrangement allows the pilot to maintain direct visual 
contact with the outside world in case of HMD system 
failures, or for scanning the cockpit instruments. Part 
of the task difficulty can be attributed to this dichoptic 
viewing situation, in which the pilot has to switch his 
attention consciously between the two eyes. 

Future research effort should be devoted to 
exploring ways to eliminate the need for maintaining 
direct visual contact with the outside world by 
incorporating all necessary information in a 
stereoscopic, full immersion display. This might 
require integrating the present cockpit panel 
information in the HMD image, and the use of 
superimposed display symbology, such as a vehicle 
path trace, a vehicle axis or velocity vector symbol. 
This superimposed symbology would serve in 
compensating for the lack of peripheral vision 
resulting from the narrow HMD field-of-view. 
Engineering efforts should be devoted primarily to 
solving the display-to-head rigidity problem, 
minimizing slaving system errors and enlarging the 
effective HMD field-of-view. 
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